1. Field of the Invention
The present invention relates generally to computer software testing, and more particularly, to methods and systems for managing an application server running a plurality of test applications.
2. Description of the Related Art
As the use of computer software in performing daily tasks is increasing rapidly, assessing software reliability through software testing has become an imperative stage in the software development cycle. As is well known, software testing is directed toward eliminating defects (i.e., bugs) in computer software, which if undetected, can create significant negative results.
As is well known, multiple groups of developers are involved during computer software application development, with each group being responsible for developing a certain portion of the application source code (i.e., a workspace). Each group is further responsible to ensure that its workspace functions properly. In achieving this task, different test applications are created to specifically test each portion of the application.
Typically, the test applications are run on the server application designed to execute the software application. The test applications are commonly submitted to the application server manually, which depending on preference, are processed serially or in parallel. Successfully processed test applications are marked as having a “pass” test result while unsuccessfully processed test applications, are marked as having a “fail” test result. A fail test result maybe generated as a result of detecting a bug in the software application being tested, in the application server processing the software application being tested, or in the test applications testing the software application.
By way of example, where test applications are processed in series, unsuccessful processing of a test application, for any reason, terminates the execution of all succeeding test applications. As a result, the generated test execution report includes a corresponding fail test result for the failed test application as well as all the succeeding test applications. Alternatively, where the test applications are being run in parallel, failure in processing one of test applications, terminates the execution of the test application, generating a test execution report that contains fail results for all of the test applications being run.
In either scenario, the generated test report includes an entry for each test application submitted for processing. That is, the test report has a corresponding pass or fail result entry for each test application fully processed. The test report further includes a corresponding fail result entry for improperly terminated test application as well as all test applications being run in parallel or subsequent to the terminated test application. Thus, one reviewing the generated test report cannot determine whether the test application failure was proper (i.e., due to a bug in the application being tested, the application server, or the test applications testing the software application) or improper (i.e., resulting from succeeding an improperly terminated test application being run concurrently with a improperly terminated test application).
Additionally, generating a complete and comprehensive test report containing results of all the test applications is impossible when the application server or network communication is down. In such scenarios, a typical test report contains a corresponding entry for each test application fully processed during the period the application server or network was functioning. However, the test report either does not include an entry for the remaining test applications or the corresponding result entries in the test report are solely failures. Again, one reviewing the test results cannot determine whether the failure of such test applications was due to a bug in the application being tested, a bug in the application server, or a bug in the test application.
In view of the foregoing, there is a need for a flexible methodology and system for managing the lifecycle of an application server running a software application being tested by a plurality of test applications.